Patient tracking using a virtual image

ABSTRACT

An apparatus and method of tracking a patient using a virtual image.

TECHNICAL FIELD

Embodiments of the present invention pertain to the field of radiationtreatment and, in particular, to patient tracking during radiationtreatment.

BACKGROUND

Radiosurgery is a minimally invasive procedure that delivers high dosesof ionizing radiation, in mono- or hypo-fractionated treatments, todestroy tumors or focal areas of pathology. The radiation dose has tooptimally fit the tumor shape, while reducing the damage to collateralorgans. The identification of the targeted lesion and its surroundingcritical tissues is typically performed in a three-dimensional (3-D)space relative to the patient's reference frame during the pre-operativelesion identification phase. During the pre-operative planning phase, aconformal dose volume is sculpted around the target while minimizing thedose delivered to adjacent healthy tissues. This may be achieved using acombination of beam positions whose relative weights or dosecontributions have been scaled to volumetrically shape the doseaccordingly. In the model known as forward planning, the user manuallyspecifies the desired weight of the various beams. The inverse planningmethod utilizes an algorithm to automatically calculate the optimumcombination of beams and weights based on user-defined dose constraintsto the target and healthy tissues.

Another method for tumor treatment is external beam radiation therapy.In one type of external beam radiation therapy, an external radiationsource is used to direct a sequence of x-ray beams at a tumor site frommultiple angles, with the patient positioned so the tumor is at thecenter of rotation (isocenter) of the beam. As the angle of theradiation source is changed, every beam passes through the tumor site,but passes through a different area of healthy tissue on its way to thetumor. As a result, the cumulative radiation dose at the tumor is highand the average radiation dose to healthy tissue is low. The amount ofradiation utilized in radiotherapy treatment sessions is typically aboutan order of magnitude smaller, as compared to the amount used in aradiosurgery session. Radiotherapy is typically characterized by a lowdose per treatment (e.g., 100-200 centi-Gray (cGy)), short treatmenttimes (e.g., 10 to 30 minutes per treatment) and hyperfractionation(e.g., 30 to 45 days of treatment). For convenience, the term “radiationtreatment” is used herein to mean radiosurgery and/or radiotherapyunless otherwise noted by the magnitude of the radiation.

During radiation treatment, a patient can change his or her position ororientation. In addition, pathological anatomies (e.g., tumor, legion,vascular malformation, etc.) may move during treatment, which decreasesaccurate target localization (i.e., accurate tracking of the position ofthe target). Most notably, soft tissue targets tend to move with patientbreathing during radiosurgical treatment delivery sessions. Respiratorymotion can move a tumor in the chest or abdomen, for example, by morethan 3 centimeters (cm). In radiation treatment, accurate delivery ofthe radiation beams to the pathological anatomy being treated can becritical, in order to achieve the radiation dose distribution that wascomputed during the treatment planning stage.

One conventional solution for tracking motion of a target utilizesexternal markers (e.g., infrared emitters) placed on the outside of apatient (e.g., on the skin). The external markers are trackedautomatically using an optical (e.g., infrared) tracking system.However, external markers cannot adequately reflect internaldisplacements caused by breathing motion. Large external patient motionmay occur together with very small internal motion. For example, theinternal target may move much slower than the skin surface.

Another conventional solution for tracking motion of a target involvesthe use of implanted fiducials. Typically, radiopaque fiducial markers(e.g., gold seeds or stainless steel screws) are implanted in closeproximity to, or within, a target organ prior to treatment and used asreference points during treatment delivery. Stereo x-ray imaging is usedduring treatment to compute the precise spatial location of thesefiducial markers (e.g., once every 10 seconds). However, internalmarkers alone may not be sufficient for accurate tracking. Furthermore,the tracking of internal fiducial markers can be difficult for thepatient, because high accuracy tends to be achieved by usingbone-implanted fiducial markers. The implanting of fiducial markers inbone requires a difficult and painful invasive procedure, especially forthe C-spine, which may frequently lead to clinical complications. Inaddition, tracking bone-implanted fiducial markers may still may notprovide accurate results for movement or deformation of soft tissuetargets. Moreover, whether the fiducial marker is implanted in the boneor injected through a biopsy needle into soft tissue in the vicinity ofthe target area under computerized tomography (CT) monitoring, thepatient must still undergo such invasive procedures before radiationtreatment.

A conventional technique that tracks the motion of a tumor without theuse of implanted fiducial markers is described in A. Schweikard, HShiomi, J. Adler, Respiration Tracking in Radiosurgery WithoutFiducials, Int J Medical Robotics and Computer Assisted Surgery, January2005, 19-27. The described fiducial-less tracking technique use imageregistration methods. These methods may differ depending on the natureof the transformation involved. In particular, the transformation can berigid or deformable. While rigid transformations (e.g., for head images)typically allow only translations and rotations, deformabletransformations require solving a significantly more complex problem.

Image registration methods can be also divided into monomodal (orintramodality) registration and multimodal (or intermodality)registration. In monomodal applications, the images to be registeredbelong to the same modality, as opposed to multimodal applications wherethe images to be registered stem from different modalities. Because ofthe high degree of similarity between the images of the same modality,solving the monomodal registration is usually an order of magnitudeeasier than in the multimodality applications, especially for deformabletransformation.

An existing approach for measuring the patient position and orientationduring radiation treatment involves registering projection X-rays takenduring treatment with a pre-treatment CT scan. However, this approach islimited because X-rays cannot be taken frequently without additionalradiation exposure to the patient. Furthermore, it is difficult to tracksoft tissue organs (e.g., lungs) on X-rays, without implanting fiducialmarkers in the vicinity of the target area.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the present invention are illustrated by way of example,and not by way of limitation, in the figures of the accompanyingdrawings and in which:

FIG. 1 illustrates one embodiment of systems that may be used inperforming radiation treatment in which features of the presentinvention may be implemented.

FIG. 2 illustrates one embodiment of an image-guided, robotic-basedradiation treatment system.

FIG. 3 is a flow diagram of one embodiment of a process for registeringimages of different modality types.

FIG. 4 is a flow diagram of one embodiment of a process for performing atreatment planning stage of image registration.

FIG. 5 is a flow diagram of one embodiment of a process for performing atreatment delivery stage of image registration.

FIG. 6 is a flow diagram of an alternative embodiment of a process forregistering images of different modality types.

FIG. 7 illustrates exemplary images used in one embodiment of adeformable registration process.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forthsuch as examples of specific components, devices, methods, etc., inorder to provide a thorough understanding of the present embodiments. Itwill be apparent, however, to one skilled in the art that these specificdetails need not be employed to practice the present embodiments. Inother instances, well-known materials or methods have not been describedin detail in order to avoid unnecessarily obscuring the presentembodiments.

The term “coupled to” as used herein may mean coupled directly to orindirectly to through one or more intervening components. Any of thesignals provided over various buses described herein may be timemultiplexed with other signals and provided over one or more commonbuses. Additionally, the interconnection between circuit components orblocks may be shown as buses or as single signal lines. Each of thebuses may alternatively be one or more single signal lines, and each ofthe single signal lines may alternatively be buses. The terms “first,”“second,” “third,” “fourth,” etc. as used herein are meant as labels todistinguish among different elements and may not necessarily have anordinal meaning according to their numerical designation.

Unless specifically stated otherwise as apparent from the followingdiscussion, it is appreciated that throughout the description,discussions utilizing terms such as “processing” or “computing” or“calculating” or “determining” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical(electronic) quantities within the computer system's registers andmemories into other data similarly represented as physical quantitieswithin the computer system memories or registers or other suchinformation storage, transmission or display devices. Embodiments of themethod described herein may be implemented using computer software. Ifwritten in a programming language conforming to a recognized standard,sequences of instructions designed to implement the methods can becompiled for execution on a variety of hardware platforms and forinterface to a variety of operating systems. In addition, embodiments ofthe present invention are not described with reference to any particularprogramming language. It will be appreciated that a variety ofprogramming languages may be used to implement embodiments of theinvention as described herein.

A method and system is described for tracking a patient during radiationtreatment using a virtual image. The term “radiation treatment” is usedherein to mean radiosurgery and/or radiotherapy unless otherwise notedby the magnitude of the radiation. The term “virtual image” used hereinrefers to an atlas, i.e., a pre-existing image of an arbitrary patienthaving substantially normal anatomy in terms of relative position andshape of structure or an averaged image of multiple patients.

During treatment planning, a required dose of radiation is determinedusing a pre-operative image (e.g., a computed tomography (CT) image, amagnetic resonance (MR) image, or fused MRI/CT). During treatmentdelivery, the current position of the patient is determined using anintra-operative image (e.g., an ultrasound (US) image, an MR image,etc.) of the patient. The intra-operative image and the pre-operativeimage may be of two different types. For example, the pre-operativeimage may be a CT image and the intra-operative image may a US image.Deformable registration of such different images creates a difficultproblem. Embodiments of the present invention overcome this difficultyby performing a series of deformable registrations using an atlas. Theatlas may be, for example, a CT atlas with ultrasound information drawnonto it, so that every position in the CT atlas has correspondingultrasound intensity. The registration between the pre-operative CTimage and the intra-operative US image may be performed, in oneembodiment, by first registering the CT image with the CT atlas using CTdata in the atlas, and then registering ultrasound data on the atlaswith the US image, thus tracking the patient position. In an alternativeembodiment, the registration may be performed by first registering theUS image with the atlas using ultrasound data on the atlas, and thenregistering the atlas with the CT image using CT data on the atlas.

FIG. 1 illustrates one embodiment of systems that may be used inperforming radiation treatment in which features of the presentinvention may be implemented. As described below and illustrated in FIG.1, systems 100 may include a diagnostic imaging system 110, a treatmentplanning system 120 and a treatment delivery system 140.

Diagnostic imaging system 110 may be any system capable of producingmedical diagnostic images of a volume of interest (VOI) in a patientthat may be used for subsequent medical diagnosis, treatment planningand/or treatment delivery. For example, diagnostic imaging system 110may be a computed tomography (CT) system, a magnetic resonance imaging(MRI) system, a positron emission tomography (PET) system, an ultrasoundsystem or the like. For ease of discussion, diagnostic imaging system110 may be discussed below at times in relation to a CT imagingmodality. However, other imaging modalities such as those above may alsobe used.

Diagnostic imaging system 110 includes an imaging source 112 to generatean imaging beam (e.g., x-rays, ultrasonic waves, radio frequency waves,etc.) and an imaging detector 116 to detect and receive the beamgenerated by imaging source 112, or a secondary beam or emissionstimulated by the beam from the imaging source (e.g., in an MRI or PETscan). In one embodiment, diagnostic imaging system 110 may include twoor more diagnostic X-ray sources and two or more corresponding imagingdetectors. For example, two x-ray sources may be disposed around apatient to be imaged, fixed at an angular separation from each other(e.g., 90 degrees, 45 degrees, etc.) and aimed through the patienttoward (an) imaging detector(s) which may be diametrically opposed tothe x-ray sources. A single large imaging detector, or multiple imagingdetectors, can also be used that would be illuminated by each x-rayimaging source. Alternatively, other numbers and configurations ofimaging sources and imaging detectors may be used.

The imaging source 112 and the imaging detector 116 are coupled to adigital processing system 114 to control the imaging operation andprocess image data. Diagnostic imaging system 110 includes a bus orother means 102 for transferring data and commands among digitalprocessing system 114, imaging source 112 and imaging detector 116.Digital processing system 114 may include one or more general-purposeprocessors (e.g., a microprocessor), special purpose processor such as adigital signal processor (DSP) or other type of device such as acontroller or field programmable gate array (FPGA). Digital processingsystem 114 may also include other components (not shown) such as memory,storage devices, network adapters and the like. Digital processingsystem 114 may be configured to generate digital diagnostic images in astandard format, such as the DICOM (Digital Imaging and Communicationsin Medicine) format, for example. In other embodiments, digitalprocessing system 114 may generate other standard or non-standarddigital image formats. Digital processing system 114 may transmitdiagnostic image files (e.g., the aforementioned DICOM formatted files)to treatment planning system 120 over a data link 118, which may be, forexample, a direct link, a local area network (LAN) link or a wide areanetwork (WAN) link such as the Internet. In addition, the informationtransferred between systems may either be pulled or pushed across thecommunication medium connecting the systems, such as in a remotediagnosis or treatment planning configuration. In remote diagnosis ortreatment planning, a user may utilize embodiments of the presentinvention to diagnose or treatment plan despite the existence of aphysical separation between the system user and the patient.

Treatment planning system 120 includes a processing device 124 toreceive and process image data. Processing device 124 may represent oneor more general-purpose processors (e.g., a microprocessor), specialpurpose processor such as a digital signal processor (DSP) or other typeof device such as a controller or field programmable gate array (FPGA).Processing device 124 may be configured to execute instructions forperforming treatment planning operations discussed herein.

Treatment planning system 120 may also include system memory 122 thatmay include a random access memory (RAM), or other dynamic storagedevices, coupled to processing device 124 by bus 128, for storinginformation and instructions to be executed by processing device 124.System memory 122 also may be used for storing temporary variables orother intermediate information during execution of instructions byprocessing device 124. System memory 122 may also include a read onlymemory (ROM) and/or other static storage device coupled to bus 128 forstoring static information and instructions for processing device 124.

Processing device 124 may also be coupled to a display device 130, suchas a cathode ray tube (CRT) or liquid crystal display (LCD), fordisplaying information (e.g., a 2D or 3D representation of the VOI) tothe user. An input device 132, such as a keyboard, may be coupled toprocessing device 124 for communicating information and/or commandselections to processing device 124. One or more other user inputdevices (e.g., a mouse, a trackball or cursor direction keys) may alsobe used to communicate directional information, to select commands forprocessing device 124 and to control cursor movements on display 130.

Treatment planning system 120 may also include storage device 126,representing one or more storage devices (e.g., a magnetic disk drive oroptical disk drive) coupled to bus 128 for storing information andinstructions. Storage device 126 may be used for storing instructionsfor performing the treatment planning operations discussed herein.

In one embodiment, storage device 126 includes a database 152 thatstores 3D images of patients obtained by the diagnostic imaging system110 prior to treatment. These images may be, for example, CT images ofpatients or MR images of patients. The treatment planning system may usethese images for radiation dose calculation and/or other treatmentplanning operations discussed herein.

In one embodiment, the treatment planning system 120 includes an atlashandling module 136 to prepare one or more atlases for future use by thetreatment planning system 120 during delivery of treatment, as will bediscussed in more detail below. In one embodiment, the atlas handlingmodule 136 resides in memory 122 and contains processing logic forexecution by the processing device 124. In another embodiment, the atlashandling module 136 contains processing logic that comprises hardwaresuch as circuitry, dedicated logic, programmable, logic, microcode, etc.In yet another embodiment, the atlas handling module 136 containsprocessing logic that comprises a combination of software and hardware.

It will be appreciated that treatment planning system 120 representsonly one example of a treatment planning system, which may have manydifferent configurations and architectures, which may include morecomponents or fewer components than treatment planning system 120 andwhich may be employed with the present invention. For example, somesystems often have multiple buses, such as a peripheral bus, a dedicatedcache bus, etc. The treatment planning system 120 may also include MIRIT(Medical Image Review and Import Tool) to support DICOM import (soimages can be fused and targets delineated on different systems and thenimported into the treatment planning system for planning and dosecalculations), expanded image fusion capabilities that allow the user totreatment plan and view dose distributions on any one of various imagingmodalities (e.g., MRI, CT, PET, etc.).

Treatment planning system 120 may share its database 152 with atreatment delivery system, such as treatment delivery system 140, sothat it may not be necessary to export from the treatment planningsystem prior to treatment delivery. Treatment planning system 120 may belinked to treatment delivery system 140 via a data link 134, which maybe a direct link, a LAN link or a WAN link as discussed above withrespect to data link 118. It should be noted that when data links 118and 134 are implemented as LAN or WAN connections, any of diagnosticimaging system 110, treatment planning system 120 and/or treatmentdelivery system 140 may be in decentralized locations such that thesystems may be physically remote from each other. Alternatively, any ofdiagnostic imaging system 110, treatment planning system 120 and/ortreatment delivery system 140 may be integrated with each other in oneor more systems.

Treatment delivery system 140 includes a therapeutic and/or surgicalradiation source 142 to administer a prescribed radiation dose to atarget volume in conformance with a treatment plan. Treatment deliverysystem 140 may also include an imaging system 144 to captureintra-treatment images of a patient volume (including the target volume)for registration or correlation with the diagnostic images describedabove in order to position the patient with respect to the radiationsource. Treatment delivery system 140 may also include a digitalprocessing system 146 to control radiation source 142, imaging system144, and a patient support device such as a treatment couch 148. Digitalprocessing system 146 may include one or more general-purpose processors(e.g., a microprocessor), special purpose processor such as a digitalsignal processor (DSP) or other type of device such as a controller orfield programmable gate array (FPGA). Digital processing system 146 mayalso include other components (not shown) such as memory, storagedevices, network adapters and the like. Digital processing system 146may be coupled to radiation source 142, imaging system 144 and treatmentcouch 148 by a bus 150 or other type of control and communicationinterface.

In one embodiment, the treatment delivery system 120 includes a patienttracking module 154 to track the patient position and orientation duringtreatment delivery. As will be discussed in more detail below, thepatient tracking module 154 may track the patient using one or moreatlases prepared by the atlas handling module 136 and stored in thedatabase 152. In one embodiment, the patient tracking module 154 residesin memory of the digital processing system 146 and contains processinglogic that is run on the digital processing system 146. In anotherembodiment, the patient tracking module 154 contains processing logicthat comprises hardware such as circuitry, dedicated logic,programmable, logic, microcode, etc. In yet another embodiment, thepatient tracking module 154 contains processing logic that comprises acombination of software and hardware.

In one embodiment, as illustrated in FIG. 2, treatment delivery system140 may be an image-guided, robotic-based radiation treatment system 200(e.g., for performing radiosurgery) such as the CyberKnife® systemdeveloped by Accuray, Inc. of California. In FIG. 2, radiation source142 may be represented by a linear accelerator (LINAC) 202 mounted onthe end of a robotic arm 204 having multiple (e.g., 5 or more) degreesof freedom in order to position the LINAC 202 to irradiate apathological anatomy (target region or volume) with beams delivered frommany angles in an operating volume (e.g., a sphere) around the patient.Treatment may involve beam paths with a single isocenter (point ofconvergence), multiple isocenters, or with a non-isocentric approach(i.e., the beams need only intersect with the pathological target volumeand do not necessarily converge on a single point, or isocenter, withinthe target). Treatment can be delivered in either a single session(mono-fraction) or in a small number of sessions (hypo-fractionation) asdetermined during treatment planning. With treatment delivery system200, in one embodiment, radiation beams may be delivered according tothe treatment plan without fixing the patient to a rigid, external frameto register the intra-operative position of the target volume with theposition of the target volume during the pre-operative treatmentplanning phase.

In FIG. 2, imaging system 144 may be represented by X-ray sources 206Aand 206B and X-ray image detectors (imagers) 208A and 208B. In oneembodiment, for example, two x-ray sources 206A and 206B may benominally aligned to project imaging x-ray beams through a patient fromtwo different angular positions (e.g., separated by 90 degrees, 45degrees, etc.) and aimed through the patient on treatment couch 148toward respective detectors 208A and 208B. In another embodiment, asingle large detector can be used that would be illuminated by eachx-ray imaging source. Alternatively, other numbers and configurations ofimaging sources and detectors may be used.

Digital processing system 146 may implement algorithms to registerimages obtained from imaging system 144 with pre-operative treatmentplanning images in order to align the patient on the treatment couch 148within the treatment delivery system 200, and to precisely position theradiation source with respect to the target volume.

The treatment couch 148 may be coupled to another robotic arm (notillustrated) having multiple (e.g., 5 or more) degrees of freedom. Thecouch arm may have five rotational and translational degrees of freedomand one substantially vertical, linear degree of freedom. Alternatively,the couch arm may have six rotational and translational degrees offreedom and one substantially vertical, linear degree of freedom or atleast four rotational and translational degrees of freedom. The coucharm may be vertically mounted to a column or wall, or horizontallymounted to pedestal, floor, or ceiling. Alternatively, the treatmentcouch 148 may be a component of another mechanical mechanism, such asthe Axum® treatment couch developed by Accuray, Inc. of California, orbe another type of conventional treatment table known to those ofordinary skill in the art.

Alternatively, treatment delivery system 200 may be another type oftreatment delivery system, for example, a gantry based (isocentric)intensity modulated radiotherapy (IMRT) system. In a gantry basedsystem, a radiation source (e.g., a LINAC) is mounted on the gantry insuch a way that it rotates in a plane corresponding to an axial slice ofthe patient. Radiation is then delivered from several positions on thecircular plane of rotation. In IMRT, the shape of the radiation beam isdefined by a multi-leaf collimator that allows portions of the beam tobe blocked, so that the remaining beam incident on the patient has apre-defined shape. The resulting system generates arbitrarily shapedradiation beams that intersect each other at the isocenter to deliver adose distribution to the target. In IMRT planning, the optimizationalgorithm selects subsets of the main beam and determines the amount oftime that the patient should be exposed to each subset, so that theprescribed dose constraints are best met.

In other embodiments, yet another type of treatment delivery system 200may be used, for example, a stereotactic frame system such as theGammaKnife®, available from Elekta of Sweden. With such a system, theoptimization algorithm (also referred to as a sphere packing algorithm)of the treatment plan determines the selection and dose weightingassigned to a group of beams forming isocenters in order to best meetprovided dose constraints.

It should be noted that the methods and apparatus described herein arenot limited to use only with medical diagnostic imaging and treatment.In alternative embodiments, the methods and apparatus herein may be usedin applications outside of the medical technology field, such asindustrial imaging and non-destructive testing of materials (e.g., motorblocks in the automotive industry, airframes in the aviation industry,welds in the construction industry and drill cores in the petroleumindustry) and seismic surveying. In such applications, for example,“treatment” may refer generally to the application of radiation beam(s).

Referring again to FIG. 1, as discussed above, the treatment planningsystem 120 may include an atlas handling module 136 to prepare one ormore atlases for future use by the treatment delivery system 140. Theatlases may include an atlas of a pre-operative imaging modality (e.g.,a CT atlas) and a corresponding atlas of an intra-operative imagingmodality (e.g., an ultrasound atlas). For example, the CT and ultrasoundatlas may be created by first performing a calibration step in which thecoordinate space of the CT scan is calibrated with respect to that ofthe ultrasound scan. One method of doing this may be to perform a CT andultrasound scan of a phantom containing fiducial points of knowngeometry that are visible in both CT and ultrasound. Another method maybe to mount the ultrasound scanner in a known orientation and positionwith respect to the CT scanner. Next, for example, the diagnosticimaging system 110 may perform a CT scan of an arbitrary patient toproduce a CT atlas, and then an ultrasound scan of the same patient,without any movement of the patient, to produce an ultrasound atlas.Alternatively, the diagnostic imaging system 110 may perform anultrasound scan of an arbitrary patient to produce an ultrasound atlas,and then a CT scan of the same patient, without any movement of thepatient, to produce a CT atlas. In one embodiment, the atlas handlingmodule 136 stores both atlases in the database 152 for future use by thetreatment delivery system 140.

In another embodiment, the atlas handling module 136 creates a combinedatlas from the two atlases described above and stores the combined atlasin the database 152 for future use by the treatment delivery system 140.In one embodiment, the atlas handling module 136 creates a combinedatlas by overlaying the first atlas with the second atlas. For example,the atlas handling module 136 may map ultrasound data from theultrasound atlas to CT data on the CT atlas, and then add the ultrasounddata to the CT atlas based on the mapping. Alternatively, the atlashandling module 136 may map CT data from the CT atlas to ultrasound dataon the ultrasound atlas, and then add the CT data to the ultrasoundatlas based on the mapping. In other embodiments, the atlas handlingmodule 136 may create a combined atlas in other manners, such as byacquiring the two atlases in the same space (e.g., using a CT/ultrasoundscanner) or by registering the first atlas with the second atlas usingtechniques known in the art.

In another embodiment, the atlas handling module 136 uses a single atlasto create a combined atlas. For example, the atlas handling module 136may select a CT atlas (e.g., a CT image of an arbitrary patient) from CTimages stored in the database 152. Then, anatomical organs may bedelineated on the CT atlas. The delineation may be performed manually,or may be aided by automated segmentation tools such as intensity-basedmethods, shape-based methods, or both. Next, Ultrasound intensities maybe added to the atlas. The ultrasound intensity for each organ may bedetermined manually by the user, or may be taken from a table stored inthe database giving typical ultrasound intensity distributions foranatomical organs. Alternatively, the atlas handling module 136 mayselect an ultrasound atlas (e.g., an ultrasound image of an arbitrarypatient) from ultrasound images stored in the database 152, determinecorresponding CT data for the positions in the ultrasound atlas usingmethods described above, and then add these corresponding CT data to theultrasound atlas. The initial atlas (e.g., a CT atlas or ultrasoundatlas) may be selected based on patient parameters provided by the user(e.g., parameters for a patient with substantially normal anatomy interms of relative position and shape of structure). Alternatively, aninitial atlas may be created by selecting images of multiple patientsand composing them into a single image pertaining to an average patient.

In one embodiment, the combined atlas is created once and is used fordifferent patients. In another embodiment, the combined atlas may becreated for each patient. In yet another embodiment, the combined atlasmay be re-created prior to each treatment delivery.

As discussed above, in one embodiment, the atlas handling module 136stores the combined atlas in the database 152 for future use by thetreatment delivery system 140. In an alternative embodiment, the atlashandling module 136 performs additional processing with respect to thecombined atlas to simplify computations during future treatmentdelivery. In particular, in one embodiment, the atlas handling module136 performs a deformable registration of a pre-operative modality image(e.g., a CT image used for radiation dose calculation) with the combinedatlas using the pre-operative modality data (e.g., CT data) on thecombined atlas. Data involved in this registration is associated withthe same modality, and, therefore, this registration is notcomputationally intense. The result of this registration is the combinedatlas deformed to match the pre-operative modality data (e.g., CT data)on the atlas with the pre-operative modality image (e.g., CT image). Theatlas handling module 136 then stores the deformed atlas in the database152 for future use by the treatment delivery system 140.

As discussed above, the treatment delivery system 140 may include thepatient tracking module 154 for tracking the patient position andorientation during treatment delivery. Small changes may be accommodatedby adjusting the parameters of the radiation source. Larger changes maynecessitate a pause in treatment while the patient is repositioned.

The patient tracking module 154 makes measurements of the patientposition and orientation by registering images obtained during treatmentwith a pre-treatment image (e.g., a pre-operative CT scan) that was usedfor treatment planning. The intra-operative images are obtained by theimaging system 144 using an imaging modality that does not requireadditional exposure of the patient to radiation. Such a modality may be,for example, an ultrasound scan or an MR scan. In one embodiment, apre-operative modality and an intra-operative modality are of twodifferent types (e.g., a pre-operative CT scan and an intra-operativeultrasound scan). Then, deformable registration may present a difficultproblem. The patient tracking module 154 solves this problem using avirtual image. In particular, in one embodiment, the patient trackingmodule 154 uses an atlas processed by the treatment planning system 120and stored in the database 152. In one embodiment, this atlas is acombined atlas that contains data of a pre-operative modality and dataof an intra-operative modality. The combined atlas may be, for example,a CT atlas with ultrasound information drawn onto it, so that everyposition in the CT atlas has a corresponding ultrasound intensity.Alternatively, this atlas may be an ultrasound atlas with correspondingCT information drawn onto it. In another embodiment, the patienttracking module 154 retrieves two atlases (e.g., a CT atlas and anultrasound atlas) from the database 152 and creates a combined atlas ina manner discussed above.

In one embodiment, the patient tracking module 154 registers apre-operative image with an intra-operative image by performing a seriesof deformable registrations. In particular, the patient tracking module154 first performs a deformable registration of the pre-operative imagewith the combined atlas using pre-operative data on the atlas, and thenperforms a deformable registration of the combined atlas with theintra-operative image using intra-operative data on the atlas. Thesedeformable registrations are intra-modality registrations and,therefore, do not involve extensive computations. The resulting deformedintra-operative image indicates changes in the patient position andorientation, as compared to the patient's position and orientation atthe time the pre-operative image was acquired.

In another embodiment, the registration of the pre-operative image withthe combined atlas is performed prior to treatment by the treatmentplanning system 120 which stores the resulting deformed atlas in theshared database 152, as discussed above. Then, the patient trackingmodule 154 retrieves the deformed atlas from the database 152, and thenregisters this atlas with the intra-operative image usingintra-operative data on the atlas.

In yet another embodiment, the patient tracking module 154 performs aseries of deformable registrations in a reversed order by firstregistering the intra-operative image with the combined atlas usingintra-operative data on the atlas, and then registering the combinedatlas with the pre-operative image. The resulting deformed pre-operativeimage then indicates changes in the patient position and orientation.

FIG. 3 is a flow diagram of one embodiment of a process 500 fordeformably registering images of different imaging modality types. Theprocess may be performed by processing logic of the atlas handlingmodule 136 and/or processing logic of the patient tracking module 154.Processing logic may comprise hardware (e.g., circuitry, dedicatedlogic, programmable logic, microcode, etc.), software (such as run on ageneral purpose computer system or a dedicated machine), or acombination of both.

Referring to FIG. 3, processing logic begins with processing logiccreating a combined atlas from two atlases of different modalities(e.g., a CT atlas and an ultrasound atlas) (processing block 502). Inone embodiment, processing logic creates the combined atlas by mappingdata from the first atlas to data in the second atlas, and thensuperimposing first modality data from the first atlas on the secondatlas using the mapping. For example, processing logic may mapultrasound data from the ultrasound atlas to CT data on the CT atlas,and then add the ultrasound data to the CT atlas based on the mapping.Alternatively, processing logic may create a combined atlas by selectinga first modality image of an arbitrary patient, automaticallydetermining second modality data corresponding to the first modalitydata in the image, and adding the second modality data to the firstmodality image.

In one embodiment, the combined atlas is created once and is used forall patients. In another embodiment, the combined atlas may be createdfor each patient. In yet another embodiment, the combined atlas may bere-created prior to each treatment delivery.

At processing block 504, processing logic obtains a first modality image(e.g., a CT image) of the patient. In one embodiment, the first modalityimage is retrieved from the database 152 to develop a treatment planbefore treatment delivery.

At processing block 506, processing logic obtains a second modalityimage (e.g., an ultrasound image) of the patient. The first and secondmodalities are of two different types. In one embodiment, the secondmodality image is obtained during treatment delivery.

Next, processing logic registers the first modality image with thesecond modality image using a series of deformable registrations. Inparticular, at processing block 508, processing logic performs adeformable registration of the first modality image with the combinedatlas using first modality data on the combined atlas. In oneembodiment, this deformable registration is performed prior to treatmentdelivery. Alternatively, it is performed during treatment delivery.

At processing block 510, processing logic performs a deformableregistration of the combined atlas deformed at processing block 508 withthe second modality image. Each of these deformable registrationsinvolves an intra-modality transformation that does not requireintensive computations. The resulting second modality image deformed tomatch the combined atlas indicates changes in the patient position andorientation.

In one embodiment, the first modality image is a pre-operative CT imageand the second modality image is an intra-operative ultrasound image,and the registration process is divided into two stages: a treatmentplanning stage and a treatment delivery stage.

FIG. 4 is a flow diagram of one embodiment of a process 600 forperforming a treatment planning stage of image registration. The processmay be performed by processing logic of the atlas handling module 136.Processing logic may comprise hardware (e.g., circuitry, dedicatedlogic, programmable logic, microcode, etc.), software (such as run on ageneral purpose computer system or a dedicated machine), or acombination of both.

Referring to FIG. 4, process 600 begins with processing logic creating acombined atlas having CT data and ultrasound data (processing block602). In one embodiment, the combined atlas is a CT atlas havingultrasound data drawn onto it, so that every position in the CT atlashas a corresponding ultrasound intensity. Alternatively, the combinedatlas is an ultrasound atlas having corresponding CT data drawn onto it.

At processing block 604, processing logic obtains a CT image of thepatient. The CT image may be obtained to identify the target andcalculate the radiation dose.

At processing block 606, processing logic performs a deformableregistration of the CT image with the combined atlas using the CT dataon the atlas.

At processing block 608, processing logic stores the combined atlasdeformed at processing block 606 in a database.

FIG. 5 is a flow diagram of one embodiment of a process 650 forperforming a treatment delivery stage of image registration. The processmay be performed by processing logic of the patient tracking module 154.Processing logic may comprise hardware (e.g., circuitry, dedicatedlogic, programmable logic, microcode, etc.), software (such as run on ageneral purpose computer system or a dedicated machine), or acombination of both.

Referring to FIG. 5, process 650 begins with processing logic obtainingan ultrasound image of the patient during treatment delivery (processingblock 652).

At processing block 654, processing logic retrieves from the databasethe combined atlas that was previously deformed to match thepre-operative CT image.

At processing block 656, processing logic performs a deformableregistration of the retrieved atlas with the ultrasound image usingultrasound data on the atlas. The ultrasound image deformed to match theultrasound data on the atlas indicates changes in the patient positionand orientation.

In an alternative embodiment, a series of deformable registrations maybe performed in a reversed order. FIG. 6 is a flow diagram of analternative embodiment of a process 700 for deformably registeringimages of different modality types. The process may be performed byprocessing logic of atlas handling module 136 and processing logic ofpatient tracking module 154. Processing logic may comprise hardware(e.g., circuitry, dedicated logic, programmable logic, microcode, etc.),software (such as run on a general purpose computer system or adedicated machine), or a combination of both. In one embodiment, theprocess 700 is performed by systems 100 of FIG. 1.

Referring to FIG. 6, processing logic begins with processing logiccreating a combined atlas (processing block 702). In one embodiment, thecombined atlas is a CT atlas having ultrasound data drawn onto it, sothat every position in the CT atlas has a corresponding ultrasoundintensity. Alternatively, the combined atlas is an ultrasound atlashaving corresponding CT data drawn onto it.

Next, processing logic obtains a CT image of the patient prior totreatment for use in treatment planning (processing block 704) andstores the CT image and the combined atlas in a database.

In one embodiment, processing logic performs blocks 702 through 706during treatment planning.

At processing block 708, processing logic obtains an ultrasound image ofthe patient during treatment delivery.

At processing block 710, processing logic retrieves the combined atlasand the pre-operative CT image of the patient from the database.

Further, processing logic registers the pre-operative CT image of thepatient with the intra-operative ultrasound image of the patient byfirst performing a deformable registration of the ultrasound image withthe combined atlas using the ultrasound data on the atlas (processingblock 712), and then performing a deformable registration of thecombined atlas with the CT image using the CT data on the atlas(processing block 714).

In one embodiment, processing logic performs blocks 708 through 714during treatment delivery.

FIG. 7 illustrates exemplary images used in one embodiment of adeformable registration process.

Referring to FIG. 7, image 802 is a pre-operative CT image that containsan anatomical structure 806. Triangle 804 represents the shape of thebody of the current patient.

Image 808 is a CT atlas that contains an anatomical structure 812. Asdiscussed above, the CT atlas may be a pre-existing CT image of this orsome other patient. Because the patient and/or the patient position isdifferent from the one associated with the CT image 802, triangle 810differs from triangle 804. Curved lines with arrows illustratedeformation of the CT atlas 808 to match data on the CT image 802.

Atlas 814 represents the CT atlas 808 with corresponding ultrasoundinformation drawn onto it. As shown, atlases 808 and 814 include commondata (anatomical structures 812 and 818), as well as someultrasound-specific data that is not present in the atlas 808(anatomical structures 816). Curved lines with arrows from the atlas 808to the atlas 814 illustrate mapping of data between the two atlases.

It should be noted that the atlas 808 is shown for illustration only andmay not be needed in the registration process. That is, the CT image 802may be registered directly with the atlas 814.

Image 822 is an intra-operative ultrasound image that has the sameanatomical structures as the atlas 814. The shape of the body of thepatient is represented by triangle 826. Curved lines with arrows fromthe atlas 814 to the ultrasound image 822 illustrate deformation of theimage 822 to match data on the atlas 814. The resulting changes on theultrasound image 822 indicate changes in the patient position andorientation as compared to the patient position and orientationreflected on the pre-operative CT image 802.

In the foregoing specification, embodiments of the invention have beendescribed with reference to specific exemplary embodiments thereof. Itwill, however, be evident that various modifications and changes may bemade thereto without departing from the broader spirit and scope of thepresent embodiments as set forth in the claims. The specification anddrawings are, accordingly, to be regarded in an illustrative rather thana restrictive sense.

1. A computer-implemented method comprising: obtaining a first image ofa first imaging modality; and registering the first image with a secondimage of a second imaging modality using at least one image atlas, thefirst and second imaging modalities being of different types.
 2. Themethod of claim 1 wherein the first and second imaging modalitiesinclude a pre-operative imaging modality and an intra-operative imagingmodality.
 3. The method of claim 1 wherein one of the first and secondimages is an ultrasound (US) image, and the other one of the first andsecond images is a computed tomography (CT) image.
 4. The method ofclaim 1 wherein one of the first and second images is a magneticresonance (MR) image.
 5. The method of claim 1 wherein one of the firstand second images is a positron emission tomography (PET) image.
 6. Themethod of claim 1 wherein each of the first and second images is athree-dimensional (3D) image.
 7. The method of claim 1 furthercomprising tracking a patient based on registering the first image withthe second image.
 8. The method of claim 1 wherein at least one imageatlas comprises a first atlas of the first imaging modality and a secondatlas of the second imaging modality.
 9. The method of claim 8 furthercomprising creating a combined atlas using the first atlas and thesecond atlas.
 10. The method of claim 9 wherein creating the combinedatlas comprises: mapping second imaging modality data from the secondatlas to first imaging modality data in the first atlas; and adding thesecond imaging modality data to the first atlas based on the mapping tocreate a combined atlas.
 11. The method of claim 9 wherein creating thecombined atlas comprises: mapping first imaging modality data from thefirst atlas to second imaging modality data in the second atlas; andadding the first imaging modality data to the second atlas based on themapping to create a combined atlas.
 12. The method of claim 9 whereinthe combined atlas is created prior to registering the first image withthe second image.
 13. The method of claim 10 wherein registering thefirst image with the second image comprises: performing a deformableregistration of the first image with the combined atlas using the firstimaging modality data in the combined atlas; and registering thecombined atlas with the second image using the second imaging modalitydata in the combined atlas.
 14. The method of claim 13 wherein: thecombined atlas is created prior to treatment; the first image isregistered with the first atlas prior to the treatment; and the firstatlas is registered with the second image during the treatment.
 15. Themethod of claim 11 wherein registering the first image with the secondimage comprises: performing a deformable registration of the first imagewith the combined atlas using the first imaging modality data in thecombined atlas; and registering the combined atlas with the second imageusing the second imaging modality data in the combined atlas.
 16. Themethod of claim 14 wherein the treatment is radiosurgical treatment. 17.The method of claim 1 wherein at least one image atlas comprises apre-existing image of an arbitrary patient having substantially normalanatomy in terms of relative position and shape of structure.
 18. Themethod of claim 1 wherein at least one image atlas comprises an averagedimage of multiple arbitrary patients.
 19. The method of claim 9 whereinthe combined atlas is created for different patients.
 20. The method ofclaim 9 wherein the combined atlas is created for each patient.
 21. Themethod of claim 9 wherein the combined atlas is re-created for eachtreatment delivery.
 22. A computer-implemented method comprising:obtaining an ultrasound (US) image of a patient during treatmentdelivery; and performing a deformable registration of the US image witha computerized tomography (CT) image obtained for the patient prior tothe treatment delivery.
 23. The method of claim 22 further comprising:determining a position of the patient during the treatment deliverybased on the deformable registration.
 24. The method of claim 22 whereineach of the US image and the CT image is a three-dimensional (3D) image.25. The method of claim 22 wherein performing the deformableregistration of the US image with the CT image comprises: performing adeformable registration of the CT image with a combined atlas using CTdata on the combined atlas; and registering the CT atlas with the USimage using US data on the combined atlas.
 26. The method of claim 25wherein: the CT image is registered with the combined atlas prior to thetreatment delivery; and the combined atlas is registered with the USimage during the treatment delivery.
 27. The method of claim 22 whereinthe treatment is radiosurgical treatment.
 28. The method of claim 26wherein the combined atlas is a CT atlas having corresponding US datasuperimposed thereon.
 29. The method of claim 26 wherein the combinedatlas is a US atlas having corresponding CT data superimposed thereon.30. An apparatus, comprising: an imaging system to provide a first imageof a first imaging modality; and a patient tracking module comprisingprocessing logic to register the first image with a second image of asecond imaging modality using at least one image atlas, the first andsecond imaging modalities being of different types.
 31. The apparatus ofclaim 30 wherein the first and second imaging modalities include apre-operative imaging modality and an intra-operative imaging modality.32. The apparatus of claim 30 wherein one of the first and second imagesis an ultrasound (US) image, and the other one of the first and secondimages is a computed tomography (CT) image.
 33. The apparatus of claim30 wherein the patient tracking module is to track a patient based onregistering the first image with the second image.
 34. The apparatus ofclaim 33 wherein at least one image atlas comprises a first atlas of thefirst imaging modality and a second atlas of the second imagingmodality.
 35. The apparatus of claim 34 further comprising an atlashandling module to create a combined atlas using the first atlas and thesecond atlas.
 36. The apparatus of claim 35 wherein: the atlas handlingmodule is to perform a deformable registration of the first image withthe combined atlas using first imaging modality data in the combinedatlas; and the patient tracking module is to register the combined atlaswith the second image using second imaging modality data in the combinedatlas.
 37. An apparatus, comprising: a diagnostic imaging system toprovide a pre-operative image of a first imaging modality; and an atlashandling module comprising processing logic to create a combined atlasfor use in registration of the pre-operative image with anintra-operative image of a second imaging modality, the combined atlashaving first imaging modality data and corresponding second imagingmodality data.
 38. The apparatus of claim 37 wherein the first andsecond imaging modalities include a pre-operative imaging modality andan intra-operative imaging modality.
 39. The apparatus of claim 37wherein one of the first and second images is an ultrasound (US) image,and the other one of the first and second images is a computedtomography (CT) image.
 40. The apparatus of claim 37 further comprising:a patient tracking module comprising processing logic to register thepre-operative image with the intra-operative image using the combinedatlas and to track a patient based on said registration.
 41. Theapparatus of claim 40 wherein: the atlas handling module is to perform adeformable registration of the first image with the combined atlas usingfirst imaging modality data in the combined atlas; and the patienttracking module is to register the combined atlas with the second imageusing second imaging modality data in the combined atlas.
 42. Amachine-readable medium containing instructions which, when executed bya processing system, cause the processing system to perform a methodcomprising: obtaining a first image of a first imaging modality; andregistering the first image with a second image of a second imagingmodality using at least one image atlas, the first and second imagingmodalities being of different types.
 43. The machine-readable medium ofclaim 42 wherein the first and second imaging modalities include apre-operative imaging modality and an intra-operative imaging modality.44. The machine-readable medium of claim 42 wherein one of the first andsecond images is an ultrasound (US) image, and the other one of thefirst and second images is a computed tomography (CT) image.
 45. Themachine-readable medium of claim 42 wherein: at least one image atlascomprises a first atlas of the first imaging modality and a second atlasof the second imaging modality; and the method further comprisescreating a combined atlas using the first atlas and the second atlas.46. The machine-readable medium of claim 45 wherein registering thefirst image with the second image comprises: performing a deformableregistration of the first image with the combined atlas using the firstimaging modality data in the combined atlas; and registering thecombined atlas with the second image using the second imaging modalitydata in the combined atlas.
 47. A machine-readable medium containinginstructions which, when executed by a processing system, cause theprocessing system to perform a method comprising: obtaining anultrasound (US) image of a patient during treatment delivery; andperforming a deformable registration of the US image with a computerizedtomography (CT) image obtained for the patient prior to the treatmentdelivery.
 48. The machine-readable medium of claim 47 wherein performingthe deformable registration of the US image with the CT image comprises:performing a deformable registration of the CT image with a combinedatlas using CT data on the combined atlas; and registering the CT atlaswith the US image using US data on the combined atlas.
 49. Themachine-readable medium of claim 48 wherein: the CT image is registeredwith the combined atlas prior to the treatment delivery; and thecombined atlas is registered with the US image during the treatmentdelivery.
 50. The machine-readable medium of claim 47 wherein thetreatment is radiosurgical treatment.